Monte Carlo simulations are generally utilized to model the outcomes of complex functions, such as those that represent large scale complex engineering systems and processes, which cannot be efficiently modeled with traditional mathematical approaches. For example, the outcomes of some complex systems or processes cannot be efficiently predicted by traditional mathematical approaches due to the inclusion of random variables and/or the introduction of uncertainty with respect to at least some of the variables. In these situations, efforts to conduct an exhaustive numerical evaluation of the function is impossible or prohibitively slow and computationally extensive. Consequently, Monte Carlo simulation may be utilized to model the probability of the different outcomes from the complex system or process in order to permit the complex system or process to be analyzed.
Monte Carlo simulations utilize random samples of the varying and uncertain parameters of the functions representative of the large scale complex engineering systems and processes in order to explore the behavior of the complex system or process. Monte Carlo simulations permit analysis of the complex systems or processes by modeling the set of possible outcomes by substituting a range of values, such as a probability distribution, for one or more of the parameters or inputs of the function being evaluated. The Monte Carlo simulation randomly samples from the distribution representative of the parameter or other input and then performs the function or set of functions representative of the complex system or process utilizing the random sample to determine an outcome from the system or process. A Monte Carlo simulation repeatedly performs this process of randomly sampling from the distributions representative of the parameters or other inputs and then performing the function or set of functions representative of the complex system or method based upon the random samples in order to effectively span the input parameter space and capture the distribution of outcomes.
While Monte Carlo simulations may vary depending upon the application, Monte Carlo simulations typically define the variables or other parameters of a function or set of functions to be evaluated as well as the probability distributions associated with one or more of the variables or other parameters. The Monte Carlo simulation then generates inputs randomly from the probability distribution of each uncertain parameter or other input. The Monte Carlo simulation performs a deterministic computation, typically termed a “run”, for a particular set of inputs in order to generate an outcome from the function or set of functions to be evaluated. The outcomes of each deterministic computation may then be aggregated to determine the statistical distribution of outcomes which may, in turn, be evaluated in order to investigate the performance of the function or set of functions representative of the complex system or process. While an effective modeling technique, Monte Carlo simulation may involve thousands or tens of thousands of deterministic computations, e.g., runs, prior to generating a representative set of outcomes that effectively model the complex system or process. The deterministic computations required for thousands or tens of thousands of runs during a Monte Carlo simulation can consume substantial computational resources and require significant time expenditure.
In order to perform a Monte Carlo simulation, the number of deterministic computations, such as generated outcomes or runs, must be defined in such a manner that the resulting set of outcomes is representative of the performance of the complex system or process. Thus, a person designing the Monte Carlo simulation advantageously has some understanding of the variation of the parameters or other inputs as well as the distribution associated with the each of the parameters or other inputs so as to estimate the number of deterministic computations required to span the input space and to generate a representative set of outcomes for the desired level of statistical confidence. In this regard, the number of samples drawn from the probability distributions associated with various parameters or other inputs must not only be sufficient to span the probability distribution of each individual parameter or other input, but also must be sufficient in number to span the combined input parameter space of the various parameters or other inputs having probability distributions associated therewith. As the number of parameters or other inputs having probability distributions associated therewith increases and/or as the variability associated with the various parameters or other inputs increases, the number of deterministic computations required for a Monte Carlo simulation to generate a set of outcomes that effectively represents a complex system or process correspondingly increases. In addition to the size of the input parameter space, complex relationships may exist between various parameters or other inputs to the function or set of functions being evaluated. These complex relationships compound the issues associated with determining the number of deterministic computations required by a Monte Carlo simulation in order to provide a statistically accurate analysis of the potential outcomes.
In instances in which the input parameter space and/or the relationship between the various parameters or other inputs to the function or set of functions being evaluated by a Monte Carlo simulation cannot be completely understood, the selection of the number of deterministic computations, that is, the number of runs, of a Monte Carlo simulation becomes more challenging and creates more risks that the resulting set of outcomes will not provide a statistically accurate portrayal of the complex system or process. Indeed, if the number of runs chosen for the Monte Carlo simulation is too small, the set of outcomes that is generated will be inaccurate and not provide a complete picture of the behavior of the complex system or process being evaluated. Conversely, if the number of runs of a Monte Carlo simulation is unnecessarily large, time and computational resources will be wasted.
For a typical Monte Carlo simulation, a user may estimate the number of runs that should be sufficient to generate a set of outcomes that permits a statistically accurate analysis of the complex system or process being modeled. Following the execution of the selected number of runs, the user may manually assess the set of outcomes to determine if additional runs should be performed. If so, the desired number of runs are performed including both the repeated performance of the runs that were previously performed as well as the performance of the additional runs and the user then again evaluates the set of outcomes. This trial-and-error method of determining the number of runs required for the set of outcomes of a Monte Carlo simulation to permit statistically accurate analysis of the complex system or process under evaluation may be time-consuming and computationally expensive. One drawback of this technique is that if additional input parameters are included, a change in the relationship between the input parameters is introduced or there is a change in variation of the input parameters, the user's a priori understanding of the minimum number of required runs is no longer valid. The reassessment of the number of runs of a Monte Carlo simulation that should be conducted in an instance in which additional or different input parameters are introduced, a change in the relationship between the input parameters is defined or there is a change in variation of the input parameters may be ignored and the user may improperly assume that the number of runs that was previously utilized for the prior Monte Carlo simulation having different input parameters or different relationships between the input parameters will also be adequate for a subsequent Monte Carlo simulation with a different number of input parameters and/or different relationships therebetween. Such an assumption is sometimes inaccurate and may result in a set of outcomes that does not permit a statistically accurate analysis of the underlying complex system or process.
The manual estimate as to the number of runs that should be conducted for an effective Monte Carlo simulation may generally result in one of four different scenarios. In a first scenario, too few runs are executed and the user concludes by analyzing the set of outcomes that additional runs are required in order to permit the complex system or process that is being modeled to be analyzed in a more statistically accurate manner. In this situation, the user determines the number of additional runs that should be conducted and the Monte Carlo simulation is again executed. This undesirably increases the time required for the Monte Carlo simulation as it must be initially commenced, the initial set of outcomes must be evaluated, the number of runs must be re-assessed and the Monte Carlo simulation must then be repeated for the additional number of runs. This iterative process may be repeated several times prior to having conducted a sufficient number of runs in order to permit a statistically accurate analysis of the complex system or method being modeled.
In a second scenario, the user also identifies an initial number of runs that are too few to generate a set of outcomes that permits the system or method to be analyzed in a statistically accurate manner. However, in this scenario, the user assumes or incorrectly assesses the initial set of outcomes and concludes that additional runs are not required. As a result of this incorrect determination, the set of outcomes does not permit a statistically accurate analysis of the complex system or process being modeled and the user may draw false conclusions from the set of outcomes. The user may be unaware of the inaccuracies and, as such, may communicate invalid conclusions relative to the system or method being modeled, thereby potentially leading to long-term programmatic consequences.
In a third scenario, the user initially identifies an excessive number of runs for the Monte Carlo simulation in an effort to insure statistically accurate results. While the set of outcomes that is generated does permit a statistically accurate analysis of the complex system or method being modeled, computational time and resources are wasted. In a fourth scenario, the user happens to identify the correct number of runs such that the set of outcomes generated by the Monte Carlo simulation permits a statistically accurate analysis of the complex system to process being modeled. In this scenario, the computational time and resource consumption are appropriate and the results are statistically accurate. However, the selection of the proper number of runs in this scenario generally does not happen very frequently and is not typically repeatable.